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1 TITLE OF THE USTS/ENTION 

2 Topographic Measurement Using Stereoscopic Picture Frames 

3 BACKGROUND OF THE INVENTION 

4 Field of the Invention 

5 The present invention relates generally to topographic measurement of 

6 a target area using image sensors mounted on flying vehicles such as earth 

7 observation satellites. 

8 Description of the Related Art 

9 Topographic measurement using stereoscopic pictures is known as 

10 remote sensing technology. In one topographic measurement system known 

11 as a cross-track stereoscopic imaging system, a single image sensor (HRV and 

12 AVNIR sensors) mounted on a satellite (SPOT and ADEOS satellites) is used 

13 to capture a number of pictures of a target area at different times when the 

14 satellite is encircling on separate orbits. In other system known as an along- 

15 track imaging system^ use is made of two image sensors (OPS and PRISM 

16 sensors) on board a single satellite (JERS-1 and ALOS satellites) to capture 

17 multiple pictures of a target area at different angles when the satellite is 

18 encircling on the same orbit over the target area. While the latter is able to 

19 send pictures at frequent intervals, hence available stereoscopic pictures can 

20 be easily obtained^ a high- capacity memory is required on board the ship to 

21 store pictures before transmission to the earth. However, in applications 

22 where a high resolution of one meter is desired, the cross-track stereoscopic 

23 imaging has been preferred to the along-track stereoscopic imaging. 

24 A recent advance in the remote sensing technology is the development 

25 • of an earth observation satellite such as IKONOS and Quick Bird satellites in 

26 which a single sensor performs the functions of both cross-track imaging and 

27 along-track imaging systems. 

28 When a pair of stereoscopic pictures is sensed, the pictures are scanned 

29 line by line and transmitted from the satellite in the form of frames to the 

30 earth station. The transmitted frames are analyzed in terms of point-to-point 
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1 correlations between the frames to determine how much they differ from one 

2 another. This correlation information is known as parallax. Using a model of 

3 the image sensor, a position is determined in a three-dimensional coordinate 

4 system for each point-to-point correlation. A set of such 3-D position data 

5 obtained from a target area constitutes topographic data of the target area. 

6 However, in order to sense a target area from an earth observation 



7 satellite^ it is necessary to ensure that when the satellite is approaching the 

8 target area, it is bright under sun light and not shadowed by any cloud. 

9 Chances for taking appropriate picttires are therefore limited. In particular, 

10 in applications where high resolution is desired, a single-sensor, cross-track 

11 earth observation satellite will be used. When the satellite is approaching a 

12 target area, the sensor must be pointed toward the target area from different 

13 angles at different times to obtain a pair of stereoscopic frames. Therefore, 

14 the target area must be clear and bright for both chances of image sensing. 

15 Additionally, the target area must be pointed from relatively large angles. 

16 This requires that the satellite orbits be distanced sufficiently from each other. 

17 During the time the satellite is encircling on intermediate orbits, no 



18 appropriate pictures cannot be taken, which leads to a low efficiency of 

19 satellite utilization. Therefore, target areas suitable for acquiring stereoscopic 

20 pictiures are significantly limited. 

21 SUMMARY OF THE INVENTION 

22 It is therefore an object of the present invention to provide a 

23 topographic data processor and a topographic measurement system which 

24 can acquire stereoscopic picture frames with high efficiency of satellite 

25 utilization. 

26 The stated object is obtained by the provision of a frame pair selector 

27 for selecting a pair of pictxire frames that constitute a stereoscopic image from 

28 multiple frames which may be stored in a storage medium or received from 

29 one or more satellites. 

30 According to a first aspect of the present invention, there is provided a 
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1 topographic data processor comprising means for selecting a pair of frames 

2 from a plurality of candidate frames of a target area captured from different 

3 high-altitude positions, the pair of frames constituting a stereoscopic image of 

4 the target area, means for determining a parallax between the selected frames 

5 and producing therefrom a first plurality of line-of-sight vectors and a second 

6 plurality of line-of-sight vectors, and means for converting the first and 

7 second pluralities of line-of-sight vectors to topographic data, 

8 A best frame pair is selected by first forming candidate picture frames 

9 into a plurality of pairs of stereoscopic frames and then evaluating the frame 

10 pairs with fitness values representative of their fitness to topographic 

11 meastirement and selecting a best frame pair having the highest fitness value. 

12 According to a second aspect, the present invention provides a 

13 topographic data processor comprising frame selecting means for selecting a 

14 pair of frames from a plurality of candidate frames of a target area captured 

15 from high-altitude positions, the selected pair of frames constituting a 

16 stereoscopic image of the target area. Scheduling means is provided for 

17 selecting at least one airborne image sensor if an appropriate frame is not 

18 available in the plurality of candidate frames and sensing picture frames from 

19 the selected image sensor, whereby the frame selecting means uses the sensed 

20 frames to select a pair of frames. A pcirallax calculation means is provided for 

21 determining a parallax between the frames selected by the frame selecting 

22 means and producing therefrom a first plurality of line-of-sight vectors and a 

23 second plurality of Une-of-sight vectors. The first and second pluralities of 

24 line-of-sight vectors are converted to topographic data, 

25 According to a third aspect of the present invention, there is provided 

26 a topographic measurement system comprising at least one image sensor 

27 mounted on a vehicle flying over a target area, a receiver for receiving a 

28 plurality of picture frames captured by the image sensor at different 

29 positions, means for selecting a pair of frames from the plurality of frames, 

30 the pair of frames constituting a stereoscopic image of the target area, meems 
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1 for determining a parallax between the selected frames and producing 

2 therefrom a first plurality of line-o£-sight vectors and a second plurality of 

3 line-of-sight vectors, and means for converting the first and second pluralities 

4 of line-of-sight vectors to topographic data. 

5 According to a fourth aspect of the present invention, there is 



6 provided a topographic measturement system comprising at least one image 

7 sensor mounted on a vehicle flying over a target area, a receiver for receiving 

8 a plurality of picture frames captured by the image sensor at different 

9 positions, frame selecting means for selecting a pair of frames from the 

10 plurality of frames of the target area, the selected pair of frames constituting a 

11 stereoscopic image of the target area, scheduling means for selecting at least 

12 one image sensor if an appropriate frame is not available in the plurality of 

13 frames, sensing picture frames from the selected image sensor, whereby the 

14 frame selecting means uses the sensed frames to select a pair of frames, 

15 means for determining a parallax between the frames selected by the frame 

16 selecting means and producing therefrom a first plurality of line-of-sight 



17 vectors and a second plurality of line-of-sight vectors, and means for 

18 converting the first and second pluralities of line-of-sight vectors to 

19 topographic data. 

20 BRIEF DESCRIPTION OF THE DRAWINGS 

21 The present invention will be described in detail further with reference 

22 to the f ollovmig drawings, in which: 

23 Fig. 1 is a schematic diagram of a land observation system according to 

24 a first embodiment of the present invention; 

25 Fig. 2 is a block diagram of a topographic data processor of the first 

26 embodiment of the present invention; 

27 Fig. 3 is a block diagram of the frame pair selector of the topographic 

28 data processor; 

29 Figs. 4 and 5 are schematic diagrams useful for describing the 

30 operation of the geometric condition analyzer of the topographic data 
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1 processor; 

2 Fig, 6 is a block diagram of the parallax calculator of the topographic 

3 data processor; 

4 Fig. 7 is a schematic diagram useful for describing the operation of the 

5 parallax calculator; 

6 Fig. 8 is a flowchart useful for describing the operation of the first 

7 embodiment of the present invention; 

8 Fig. 9 is a block diagram of a topographic data processor of a second 

9 embodiment of the present invention; 

10 Fig. 10 is a block diagram of an image sensing scheduler of Fig. 9; 

11 Fig. 11 is a schematic diagram usef\U for describing the operation of a 

12 geometric condition analyzer of Fig, 10; and 

13 Fig. 12 is a flowchart useful for describing the operation of the second 

14 embodiment of the present invention. 

15 DETAILED DESCRIPTION 

16 Referring now to Figs. 1 and 2, there is shown a land observation 



17 system according to a first embodiment of the present invention. The system 

18 includes a first land observation satellite 11 encircling the earth on an orbit 16 

19 and a second land observation satellite 12 encircling the earth on an orbit 17 

20 spaced from the orbit 16. Each satellite is constantly capturing images of the 

21 land surface including a target area 13. Specifically, the satellite 11 includes a 

22 two-dimensional CCD sensor 21 and a high-precision telescope, not shown, 

23 for focusing the image of a land surface onto the CCD sensor 21 and an RF 

24 transmitter 23- Likewise, the satellite 12 includes a two-dimensional CCD 

25 sensor 22 and a high-precision telescope, not shown, for focusing the image 

26 of a land surface onto the CCD sensor 22 and an RF transmitter 24. 

27 The images (pictiu'e frames) captured by both satellites are tr2Uismitted 

28 on a downlink frequency from the transmitters 23 and 24 to an earth station 

29 14 where the RF signals are amplified and down-converted to baseband 

30 signals by an RF transceiver 25. The signals received from both satellites are 
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1 stored in a stotage medium 27 which may be provided in a topographic data 

2 processor 15. In this way, the storage medium 27 stores a series of picture 

3 frames of land surfaces captured by satellites 11 and 12. 

4 As will be described in detail later, a pair of picture frames which form 

5 a stereoscopic image adequate for the determination of the altitude of a 

6 surface feature of a target area is selected by a frame pair selector 28 from the 

7 storage medium 27. The selected pair of picture frames are supplied to a 

8 parallax calculator 29 to detect a parallax between the selected frames. Based 

9 on the calculated parallax, the parallax calculator 29 produces a first plurality 

10 of line-of-sight vectors from one of the selected frames and a second plurality 

11 of line-of-sight vectors from the other frame. A parallax-terrain converter 30 

12 is connected to the output of the parallax calculator 29 to produce 

13 topographic data based on the first plurality of line-of-sight vectors and the 

14 second plurality of line-of-sight vectors. The topographic data is supplied to 

15 output means 31. 

16 As illustrated in detail in Fig. 3, the frame pair selector 28 comprises a 

17 decision modtde 40^ a frame combiner 41 and a plurality of analyzers 42 to 46. 

18 Since the picture frames captured by the satellites may contain images 

19 which cover outside of the target area 13, the frame combiner 41 first selects 

20 only those picture frames covering the target area and proceeds to combine 

21 picture frames selected from those captured by satellite 1 1 with frames 

22 selected from those captured by satellite 12 to form a plurality of frame pairs 

23 each composing a stereoscopic image of the target area. If a frame consists of 

24 red, green, blue and near-infrared light components, these components are 

25 not treated individually in so far as their resolutions are equal to each other. 

26 Since panchromatic images have in most cases twice as high resolution as 

27 red, green and blue components, they are treated separately from the color 

28 components. All pairs of frames combined by the frame combiner 41 are 

29 supplied to all analyzers 42 to 46 as candidate frame pairs. 

30 Analyzers 42 to 46 perform individual analysis on the candidate frame 
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1 pairs according to different visual characteristics and evaluate each pair of 

2 frames with a set of fitness values Ql through Q6 each being representative 

3 of the fitness of the frame pair to topographic measurement of the target area. 

4 For each frame pair^ the decision module 40 scales the fitness values 

5 with respective weight values wj and totals the weighted fitness values w^Qj 

6 to produce a quality value Q of the frame pair. The process is repeated on all 

7 frame pairs and finally the decision module 40 selects one of the frame pairs 

8 having the highest quality value as a best frame pair. 

9 Analyzer 42 is a geometric condition analyzer which evaluates the 

10 candidate frame pairs and determines a weight value Ql according to 

11 geometric conditions of the frame of each pair, such as the resolution (i,ev an 

12 area covered by a single pixel and measured in terms of meters) or the angles 

13 of orientation of the satellites to the land surface. In a simple yet effective 

14 method, a pair of high-resolution frames is evaluated with a high weight 

15 value. 



16 If the frames of a pair are of different resolution, the frame of lower 

17 resolution is chosen to evaluate the pair. The evaluation of a pair with lower 

18 resolution is preferred to evaluating the pair with an average value of the 

19 resolutions of its frames. In this case, the fitness value Ql is given as: 

20 Ql = 1/ Resolution (1) 

21 If high precision is important for the frame pair evaluation, quantum 

22 errors of the frames of each pair are used to evaluate the pair. In principle, 

23 this method involves determining a displacement in a three-dimensional 

24 space between the frames of a stereoscopic pair on a pixel-by-pixel basis and 

25 representing it a quantum error in a system of three-dimensional axes. 

26 Assume that the image sensors of satellites 11 and 12 are pointing 

27 toward a target point "o'' in a three-dimensional coordinate system (x, y, z) 

28 with its X and y axes pointing North and East, respectively, on the earth 

29 surface 18 (which is approximated as a flat plane), and its z axis pointing 

30 skyward, as shown in Fig. 4. The image sensor of satellite 11 is located in a 
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1 position of altitude from the ground point gj, azimuth angle aj and 

2 angle of elevation e^^ while the image sensor of satellite 12 is located in a 

3 position $2 of altitude h2 from the ground point g2/ azimuth angle a2 and 

4 angle of elevation e2. Ground points gi and g2 are at distances bj and b2 

5 from the point of origin o, respectively, and mutually spaced at distance 

6 Points o, gi and g2 form a triangle with inner angles sty a4 and and points 

7 o^ S| and S2 form a triangle, called epipolar plane 50. A portion of the 

8 epipolar plane 50 in the neighborhood of the point of origin "o^' can be 

9 enlarged as shown in Fig. 5 to illustrate its details to the size of pixels. 

10 In Fig« 5, adjacent pixels on the image sensor of satellite 11 are 

11 indicated as pn and p;^2 fro^ which line-of-sight vectors and vi2 extend 

12 toward the point of origin. Likewise, adjacent pixels on the image sensor of 

13 satellite 12 are indicated as P2i and p22 from which line-of-sight vectors V21 

14 and V22 extend toward the point of origin. Horizontal distance r^ between 

15 line-of-sight vectors v^ and V12 represents the resolution of the image sensor 

16 of satellite 11 and horizontal distance r2 between line-of-sight vectors V21 and 

17 V22 represents the resolution of the image sensor of satellite 12. 



18 On the epipolar plane 50, the line-of-sight vector V12 forms an angle e^' 

19 to the earth surface 18 and the line-of-sight vector V22 forms an angle 62' to 

20 the earth surface 18, The angles ej', 62' and the resolutions r^, r2 ai« given as 

21 follows: 

22 e^' =a arctan {hj / bj cos a3} (2a) 

23 ^2 = arctan [h2/ b2 cos 34} (2b) 

24 = Ti sin ej' (3a) 

25 r2 = r2' sin e2 (3b) 

26 where r^' and r2' are the resolutions of the sensors of satellites 11 and 12 

27 when they directly point to ground points gj and g2/ respectively. 

28 The quantum error at the point of origin "o'^ is represented by an area 

29 50A defined by line segments C1-C2/ C2-C3, C3-C4 and C4-cj^. Therefore, a 

30 stereoscopic image whose area 50A is small is evaluated with a high fitness 
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1 value since the size of the area 50A determines the resolution of the image. 

2 The quantum error is decomposed into a horizontal component Ej^ and a 

3 vertical component which are given as follows: 

4 Eh = (ri' tan ei' + tan €2') / (tan ei' + tan ©2') (4a) 

5 Ev = (ri' + (tan ei') (tan 62') / (tan e^' + tan 62') (4b) 

6 It is seen that the fitness value Ql of a frame pair is inversely 

7 proportional to its quantum error as given by the following relations: 

8 Ql =s 1 (if Ej^ < E^^req and Ey < Ey_req) 

9 =0 (otherwise) (5) 

10 where E^-req and Ev_req are threshold values of Ej^ and Ey, 

11 Alternatively, the fitness value Ql can be determined as: 

12 Ql = q„l/Eh + 1/Ev (6) 

13 where q_l represents a parameter of positive value for giving different 

14 weights to the error components E^ and Ey. hi most cases, the parameter 

15 q_l is equal to unity. In comparison with other analyzers, the geometric 

16 condition analyzer 42 plays a significant role for selecting a best frame pair 

17 from the candidate frame pairs. 

18 Analyzer 43 is a filtering condition analyzer which evaluates the frame 

19 pairs from the frame combiner 41 in terms of their weight to topographic 

20 measurement which varies depending on the filtering characteristics of the 

21 satellite image sensors. Picture frames obtained by a satellite image sensor 



22 with the full spectnmi of visible lights i.e., panchromatic images, are usually 

23 of high S/N quality and fitting to topographic measurement. If two frames 

24 are obtained by image sensors of like filtering characteristics, they are also 

25 suitable to form a pair for topographic measurement since they are less 

26 affected by differences in filtering characteristics when a parallax is calculated 

27 between them. Therefore, the filtering condition analyzer 43 evaluates the 

28 frames of each candidate pair with a fitness value Q2 which is proportional to 

29 the amount of visible spectral components they have obtained as a resiilt of 

30 the filtering characteristics of the satellite sensors as well as to the likeness of 
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their wavelength characteristics to each other. The fitness value Q2 is 
expressed by the foliowing formulat 



where, S indicates the spectrum of visible hght w is the wavelength, and f(w) 
and g(w} represent the filtering characteristics of the image sensors of 
satellites 11 and 12, respectively (i.e., the transmissibility of filtered visible 
wavelengths incident on the image sexisors). The functions f (w) and g(w) are 
of high value if the transmissibility of wavelengths is high. Equation (7) thus 
indicates that greater the filtering functions overlap each other the fitness 
value Q2 becomes higher. The panchromatic image is given the highest Q2 
value. 

Analyzer 44 is a sunlight condition analyzer which evaluates the 
frames of each candidate pair with a fitness value Q3 in temts of their weight 
to the calculation of peirallax which varies depending on the sunlight 
condition under which the frames are captured. If sunlight conditions under 
which frames of a pair are captured are substantially equal to each other, 
their images will show similar shadow and shading effects to each other. 
Since the frames of like sunlight conditions result in an accurate parallax, 
they are evaluated with high fitness value Q3. 

The following is an evaluation formula for the fitness value Q3: 

Q3 = 1 (if I ai - 32 I < d and I e^ - 62 I < d) 



where, d is a threshold angle of several degrees. Equation (8) indicates that 
Q3 is high if azimuth angles aj and a2 and angles of elevation e^ and e2 
almost equal to each other. If their differences exceed the threshold angle, the 
influence of stmlight conditions on frames remains constant. 

Analyzer 44 is a time-difference analyzer which evaluates the frame 
pairs from the frame combiner 41 in terms of their weight to topographic 
measurement which varies depending on the time difference between the 
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1 frames of each pair. Since the time difference between frames (two or three 

2 seconds, for example) may result in objects pictured in one frame taking 

3 different shapes and positions from those of the other, a high fitness value Q4 

4 is given to a pair of frames whose time difference is smalL However, the time 

5 difference may vary significantly depending on the orbits of satellites and 

6 other factors. 

7 Q4 is given as follows: 

8 Q4 = exp (-Ta) (9) 

9 where is the absolute value of difference between the pictured timings of 

10 the frames and represented by the number of days. The fitness value Q4 may 

11 also be given by observing frame pictures by human eyes. If the pictures of 

12 any two frames differ substantially, a low value of Q4 is manually given to 

13 these frames. 

14 Analyzer 46 is a frame matching analyzer that evaluates each pair of 

15 combined frames with a fitness value Q5 in terms of their degree of match 

16 between the combined frames. The degree of match between frames of each 

17 pair is determined by an average value of correlation values obtained by the 

18 paraUax calculator 29 for each frame pair as follows; 

19 QS^AVs (10) 

20 As will be described later, the parallax calculator 29 defines windows 

21 (the size of a few pixels) in corresponding positions of the frames of a pair 

22 and obtains correlation values between the windows. Frame matching 

23 analyzer 46 averages the correlation values (between -1 and 1) obtained by 

24 the parallax calculator 29. If the average value of correlations between frames 

25 of a pair is high, it is considered that there is a high degree of match between 

26 the frames and the frame pair is evaluated with a high fitness value Q5. 

27 It is seen from the foregoing that, for each frame pair, a set of fitness 

28 values Ql ^ Q5 is obtained. The same process is repeated imtil a plurality of 

29 sets of fitness values Ql - Q5 are obtained f rorn all frame pairs formed by the 

30 frame combiner 41 . 
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1 Specifically, the decision module 40 calculates the following Equation 

2 by respectively weighting the fitness values of a frame pair to obtain a total 

3 value of weighted fitness value as a quality value Q of the frame pair: 

4 Q = wjQl + W2Q2 + W3Q3 + W4Q4 + W5Q5 (11) 

5 where, wj through W5 are weight values for the analyzers 42 through 46. 

6 One example of the weight values is: 

7 wj = 10 and W2 = W3 = W4 = W5 = 1 

8 If the frame pair selector 28 is provided with only one analyzer, Le,, 

9 the geometric condition analyzer 42, the weight value v^i is set equal to unity 

10 and all other weight values are set equal to zero. 

11 Decision module 40 repeats the same process on all frame pairs 

12 supplied from aU analyzers to produce their quality values Q and selects one 

13 of the frame pair having the highest quality value, 

14 As shown in Fig. 6, the parallax calculator 29, connected to the output 

15 of the frame pair selector 28^ includes an interpolator 61, a frame aligner 62 

16 and a correlation calculator 63. If frames of different resolutions are selected 

17 as a best pair by the frame pair selector 28, the frame of lower resolution is 

18 supplied to the interpolator 61 to improve its resolution so that it is equal to 

19 the resolution of the other frame. The frames are then supplied to the frame 

20 aligner 62, where the frames are aHgned so that they are parallel with an 

21 epipolar line of a stereoscopic image. The aligned frames are fed into the 

22 correlation calculator 63, 

23 As shown in Fig. 7, the correlation calculator 63 segments each of the 

24 aligned frames into rectangular windows of several pixels on each side and 

25 takes correlation between windows A and B of corresponding positions to 

26 determine a correlation value r; as follows: 

27 . ' 

j.k) - M(A))(l(B, j,k) - M(B))} 

28 n k = , — , . , = ^ (12) 

lXX{l(A/j,k)«M(A)2} £S{l(B,j,k)-M(B)2} 
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1 where, I( A, j, k) is the pixel value of point (j, k) of the window A, I(B, j, k) is 

2 the pixel value of point (j, k) of the window and M(A) is the average of 

3 pixel values of window A, and M(B) is the average of pixel values of window 

4 B. 

5 By successively shifting the positions of the windows A and B pixel by 

6 pixel and calculating correlation values^ the analyzer 46 seeks relative 

7 positions of windows A and B where there is a peak or ntaximum correlation 

8 value. When such positions are detected, the analyzer 46 produces output 

9 data indicating the corresponding relationship between the center pixel of 

10 window A and the center pixel of window B. The above process is repeated 

11 for all windows of the frames to obtain a plurality of frame-to-frame 

12 corresponding relationships (i.e., line-of-sight vectors) as parallax data, 

13 On the other hand, the parallax data are obtained from all frame pairs 

14 supplied from the frame pair selector 28 are fed back to the frame matching 

15 analyzer 46. When this occurs, the decision module of the frame pair selector 

16 28 receives all sets of weight data Ql to Q5 from the analyzers 42 to 46 and 

17 selects a best frame pair and commands the correlation calculator 63 to 

18 supply the parallax data of the best frame pair to the parallax-terrain 

19 converter 30* 

20 In more detail, as previously described with reference to Fig* 5, if 

21 pixels and P22 correspond to each other, the intersection point Cj of line- 

22 of-sight vectors v^^ and V22 lies on a surface feature of the target area. In 

23 order to obtain a plurality of such intersection points in a three-dimensional 

24 coordinate system to produce topographic data, this conversion process is 

25 repeated for all line-of-sight vectors represented by the parallax data to 

26 describe all surface features of the target area. 

27 Parallax-terrain converter 30 performs a parallax-terrain conversion 

28 process by using the line-of-sight vectors indicated by the parallax data of the 

29 selected frame pair to produce terrain data which represents surface features 

30 of the target area. 
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1 Topographic data is supplied from the parallax-terrain converter 30 to 

2 output means 31 such as display or memory, 

3 Due to the provision of the frame pair selector 28, picture frames taken 

4 by any pair of multiple image sensors can be selected for producing 

5 topographic data^ eliixiinating the inability to produce topographic data due 

6 to the absence of stereoscopic images and enabling the selection of best 

7 stereoscopic images for particular purposes. 

8 Fig. 8 is a flowchart which summarizes the operation of the first 



9 embodiment of the present invention. At step 81, the satellites 11 and 12 are 

10 encircling the earth to capture photographic images of various regions of the 

11 earth on predetermined schedule. The captured images are scanned and 

12 transmitted to the earth station 14 and stored in storage medium 27 (step 82). 

13 In response to demand for creating topographic data, a plurality of frame 

14 pairs containing a target area are selected from the storage medium (step 83). 

15 Parallax is calculated between the frames of each selected pair (step 84). 

16 When parallax is calculated for all frame pairs, a best frame pair is selected 

17 (step 85). Line-of-sight vectors indicated by the parallax data of the best 

18 frame pair are used to produce topographic data (step 86) and utilized for 

19 display, storage or transmission (step 87), 



20 In the first embodiment of this invention, topographic data is 

21 produced exclusively from the stored data in the storage medium 27. If 

22 appropriate frames are not available in the storage medium, no topographic 

23 data is obtained. 

24 A second embodiment of the present invention^ shown in Fig. 9, is 



25 intended to solve this problem. In Fig. 9, parts corresponding to those in Fig. 

26 2 are marked with the same numerals and the description thereof is omitted. 

27 In this embodiment, the system additionally includes an image sensing 

28 schedtder 90 which is connected to tlie frame pair selector 28 and to the RF 

29 transceiver 25. The decision module of frame pair selector 28 checks to see if 

30 there is no usable frame or if the parallax data of the best frame pair is not 
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1 usable, In either case, the frame pair selector 28 instructs the image sensing 

2 scheduler 91 to proceed with the formulation of an image sensing schedule. 

3 According to the formulated schedule^ the scheduler 90 sends a sensing 

4 command signal to one or more earth observation satellites 91 via the RF 

5 transceiver 25. 

6 As shown in Fig, 10, the scheduler 90 includes a decision module 100^ a 

7 satellite selector 101, a geometric condition analyzer 102, a filtering condition 

8 analyzer 103, a sunlight condition analyzer 104, a time-difference analyzer 

9 105 and a frame matching analyzer 106. 

10 Satellite selector 101 is responsive to the instruction from the frame 

11 pair selector 28 to select one or more earth observation satellites which cover 

12 the target area and send picture frames within a scheduled interval of time, 

13 which may be ten minutes or as long as several months. If a sateUite is flying 

14 over the same area several times during a known time interval, the image 

15 sensor of the same sateUite is treated as a sepeirate sensor as long as the 

16 pictures are captured at different angles to the target area. According to the 

17 instruction from the frame pair selector 28, the satellite selector 101 receives 

18 picture frames of the selected satellites from the RF transceiver 25 and 

19 supplies the received frames to the modules 102, 103, 104 and analyzers 105, 

20 106. 

21 Geometric condition analyzer 102 combines the picture frames 

22 supplied from the satellite selector 101 into a plurality of pairs of frames that 

23 compose stereoscopic images and supplies the pairs of combined frames to 

24 the other analyzers 103 to 106. Analyzer 102 further informs the decision 

25 module 100 of the identifiers of the satellites from which the paired picture 

26 frames are obtained. As described previously with respect to the geometric 

27 condition analyzer 42, the geometric condition analyzer 102 calculates the 

28 quantxim errors of the combined frames of each stereoscopic pair and assigns 

29 a fitness value Q6 to each frame pair so that a highest value Q6 is given to a 

30 frame pair of smallest quantum error. 
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1 The operation of the geometric condition analyzer 101 for detecting 

2 quantum errors between two picture frames will be described with reference 

3 to Fig. 11. 

4 Assume that satellites 11 and 12 are selected by the satellite selector 



5 101, respectively encircling the earth on orbits 16 and 17 that cover the target 

6 area 13. A first set of sensing points p^.j - pi.5 are established at intervals 

7 along the orbit 16, and a second set of sensing points P2-1 

8 established at intervals along the orbit 17. Picture frames at the sensing 

9 points are then paired between the first and second sets, such as between 

10 point P1.3 and point p2.i, for example. For each pair of sensing points, a 

11 quantum error is then calculated between the frames at the sensing points 

12 and the pair of sensing points is evaluated with a fitness vdue Q6 as follows: 



13 Q6 = q.6/EhH-l/Ev (13) 

14 where q_6 is a positive weight value which is usually equal to unity. High 

15 Q6 value is assigned to a frame pair if the horizontal and vertical components 

16 of the quantum error are small. 

17 If more than two satellites are selected, sensing points are established 

18 in the same manner as discussed above. If only one new picture frame is 

19 desired, using a stored frame as its companion. Equation (13) is calculated by 

20 assuming that there is only one fixed sensing point for one of the satellites. 

21 Filtering condition analyzer 103, the sunlight condition analyzer 104, 



22 the time difference analyzer 105 and the frame matching analyzer 106 

23 correspond respectively to the filtering condition analyzer 43^ sunlight 

24 condition analyzer 44, time difference analyzer 45 and frame matching 

25 analyzer 45, and operate in like manner to that described previously to 

26 produce fitness values Q7, Q8, Q9 and QIO. 



27 Decision module 100 makes a decision on the fitness values Q6 - QIO 

28 of each frame pair and produces a total of weighted fitness values (quality 

29 value) Q of the frame pair as follows: 

30 Q = wg Q6 + W7 Q7 + wg Q8 + W9 Q9 + wjq QIO (14) 
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1 where, ^'^lo weight values of the corresponding fitness values Q6 to 

2 QIO. If use is made of only the geometric condition analyzer 102, the weight 

3 value W5 is set equal to 1 and all the other weight values are set to zero, 

4 Based on the total fitness value Q, the decision module 100 formulates 

5 an image sensing schedule. The schedule includes data identifying satellites 

6 to be used, sensing positions, sensing times and filtering conditions. 

7 According to the schedule, the decision module 100 sends a command signal 

8 to one or more satellites through the transmitter 92. 

9 Fig. 12 is a flowchart which summarizes the operation of the second 

10 embodiment of the present invention. At step 12 1^ frame pair selection is 

11 performed to select a plurality of pairs of frames from the storage medium 27. 

12 At decision step 122^ decision is made as to whether or not usable frames are 

13 available. If the selected frames are usable for producing topographic data, 

14 flow proceeds to step 123 to perform parallax calculation and select a best 

15 frame pair. At step 124, decision is made as to whether or not parallax data of 

16 the best frame pair is usable. If the decision is affirmative, flow proceeds to 

17 step 125 to perform parallax- terrain conversion to produce topographic data 

18 which is output to a display or the like (step 126). 

19 If the decision at step 122 or 124 is negative, flow proceeds to step 127 

20 to formulate an image sensing schedule and transmit a command signal to 

21 one or more satellites (step 128) and retiims to step 121. 
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